WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCX 

INTERNATIONAL APPUCATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification ^ : 
B60Q m4 



Al 



(11) International PubUcation Number: WO 97/JS743 

(43) International PubUcation Date: 2 October 1997 (02.10.97) 



(21) International AppUcation Number: PCT/US97/04829 

(22) IntemaUonal Filing Date: 25 Niarch 1997 (25.03.97) 



(30) Priority Data: 

08/62 U863 



25 March 1996 (25.03,96) 



US 



(71) Applicant: DONNELLY CORPORATION [US/US]; 414 East 

Fortieth Street, Holland, MI 49323 (US). 

(72) Inventors: SCHOFIELD. Kenneth; 4793 Crestridge Court, 

Holland. MI 49423 (US). LARSON. Mark, L.; 13873 
Oakwood Circle. Grand Haven. MI 49417 (US). VADAS. 
Keith, J.; 552 Harrison Avenue, Coopcrsville, MI 49404 

(US). 

(74) Agent: BURKHART, Frederick, S.; Van Dyke. Gardner, Linn 
& Burkhart. LL.P., Suite 207. 2851 Charlevoix Drive, S£., 
Grand Rapids. Ml 49546 (US). 



(81) Designated States: AL. AM. AT. AU. AZ. BB. BG. BR, BY. 
CA. CH. CN. CZ. DE. DK. EE. ES. H. GB. GE, HU. 
IS. JP. KE. KG. KP, KR. KZ, LK. LR. LS. LT, LU, LV. 
MD. MG. MK, MN. MW. MX, NO, NZ. PL, PT, RO, RU. 
SD, SE, SG. SI. SK. TJ, TM. TR, TT. UA. UG, UZ. VN. 
ARIPO patent (GH. KE. LS. MW, SD, SZ. UG), Eurasian 
patent (AM. AZ, BY. KG, KZ. MD, RU. TJ. TM). European 
patent (AT, BE, CH. DE, DK. ES. R. FR, GB, GR, IE, IT. 
LU, MC. NL. PT. SE), OAPI patent (BF. BJ. CF, CG. CI. 
CM. GA. GN, ML, MR. NE, SN. TD, TG). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Tide: VEHICLE HEADLIGHT CONTROL USING IMAGING SENSOR 
(57) Abstract 



A vehicle headlamp control method and appa- 
ratus includes providing an imaging sensor (14) that 
senses li^t in spatially separated regions of a field of 
view forward of the vehicle. Light levels sensed in in- 
dividual regions of the field of view by a sensor array 
(38) arc evahiated in order to identify light sources of 
interest, such as oncoming headlights and leading tail- 
lights. 'Hie vehicle's headlights (18) are controlled by 
lighting control logic (16) in response to identifying 
such particular light sources or absence of such light 
sources. Spectral signatures of light sources may be 
examined in order to determine if the spectral signa- 
ture matches that of particular light sources such as the 
spectral signatures of headlights or tailllghts. Sensed 
light levels may also be evaluated for their spatial dis- 
tributicms in order to identify light sources of interest. 
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VEHICLE HEA DLIGHT CONTRO T. nsnsin imahtno sfmsor 
CROSS-RRFRRFNCF TO RFT ^JFP APPT irATTOM 
This application is a continuation-in-part of co-pending application Serial No. 
08/023,918 filed February 26, 1993. by Kenneth Schofield and Mark Larson. 

BACKGROT INn QP THF TMVFNTT Q N 
This invention relates generally to vehicle control systems and, in particular, to a system 
and method for controlling the headlights of the vehicles. The invention is particularly adapted 
to controlling the vehicle's headlamps in response to sensing the headlights of oncoming 
vehicles and taillights of leading vehicles. 

It has long been a goal to automatically control the state of a vehicle's headlights in order 
to accomplish automatically that which is manually performed by the driver. In particular, the 
driver of a vehicle whose headlights are in a high-beam state will dim the headlights upon 
conscious realization that the headlights are a distraction to the driver of an oncoming vehicle or 
a leading vehicle. It is desirable to relieve the driver of such duties and thereby allow the driver 
to concentrate on tiie driving task at hand. The ideal automatic control would also facilitate the 
use of high beams in conditions which allow their use, increasing the safety for the controlled 
vehicle as well as reducing the hazard caused by the occasional failure of the driver to dim the 
headlights when such headlights are distracting another driver. 

Prior attempts at vehicle headlight dimming controls have included a single light sensor 
which integrates light in the scene forward of the vehicle. When the integrated light exceeds a 
threshold, tiie vehicle headlights are dimmed. Such approaches have been ineffective. The 
headlights of oncoming vehicles are, at least from a distance, point sources of light. In order to 
detect such light sources in an integrated scene, it is necessary to set a sufficientiy low threshold 
of detection tiiat many non-point-sources at lower intensities are interpreted as headlights or 
taiUights. Such prior art vehicle headlight dimming controls have also been ineffective at 
reliably detecting the taUlights of leading vehicles. The apparent reason is that the 
characteristics of tiiese two light sources; for example, intensity, are so different tiiat detecting 
both has been impractical. In order to overcome such deficiencies, additional solutions have 
been attempted, such as tiie use of infrared filtering, baffling of Uie optic sensor, and tiie like. 
While such modifications may have improved performance somewhat, tiie long-felt need for a 
commerciaUy usefiil vehicle headlight dimming control has gone unmet 
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SUMMARY OF THF INVFNTTON 

The present invention provides a vehicle control which is capable of identifying unique 
characteristics of light sources based upon a precise evaluation of light source characteristics 
made in each portion of the scene forward of the vehicle, in the vicinity of each light source, by 
5 separating each light source from the remainder of the scene and analyzing that source to 

determine its characteristics. One characteristic used in identifying a light source is the spectral 
characteristics of that source which is compared with spectral signatures of known light sources, 
such as those of headlights and taillights. Another characteristic used in identifying a light 
source is the spatial layout of the light source. By providing the ability to identify the headlights 

10 of oncoming vehicles and the taillights of leading vehicles, the state of the headlights of the 

controlled vehicle may be adjusted in response to the presence or absence of either of these light 
sources or the intensity of these light sources. 

This is accomplished according to an aspect of the invention by providing an imaging 
sensor which divides the scene forward of the vehicle into a plurality of spatially separated 

1 5 sensing regions. A control circuit is provided that is responsive to the photosensors in order to 
determine if individual regions include light levels having a particular intensity. The control 
circuit thereby identifies particular light sources and provides a control output to the vehicle that 
is a function of the light source identified. The control output may control the dimmed state of 
the vehicle's headlamps. 

20 In order to more robustly respond to the different characteristics of headlights and 

taillights, a different exposure period is provided for the array in order to detect each light 
source. In particular, the exposure period may be longer for detecting leading taillights and 
significantly shorter for detecting oncoming headlights. 

According to another aspect of the invention, a solid*state light imaging array is provided 

25 that is made up of a plurality of sensors arranged in a matrix on at least one semiconductor 

substrate. The light-imaging array includes at least one spectral separation device, wherein each 
of the sensors responds to light in a particular spectral region. The control circuit responds to 
the plurality of sensors in order to determine if spatially adjacent regions of the field of view 
forward of the vehicle include light of a particular spectral signature above a particular intensity 

30 level. In this manner, the control identifies light sources that are either oncoming headlights or 
leading taillights by identifying such light sources according to their spectral makeup. 
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According to another aspect of the invention, a solid-state light-imaging array is provided 
that is made up of a plurality of sensors that divide the scene forward of the vehicle into spatially 
separated regions, and light sources are identified, at least in part, according to their spatial 
distribution across the regions. This aspect of the invention is based upon a recognition that 
headlights of oncoming vehicles and taillights of leading vehicles are of interest to the control, 
irrespective of separation distance from the controlled vehicle, if the source is on the central axis 
of travel of the vehicle. Oncoming headlights and leading taillights may also be of interest away 
from this axis, or off axis, but only if the source has a higher intensity level and is spatially 
larger. These characteristics of headlights and taillights of interest may be taken into 
consideration by increasing the resolution of the imaging airay along this central axis or by 
increasing the detection threshold off axis, or both. Such spatial evaluation may be implemented 
by selecting characteristics of an optical device provided with the imaging sensor, such as 
providing increased magnification central of the forward scene, or providing a wide horizontal 
view and narrow vertical view, or the like, or by arrangement of the sensing circuitry, or a 
combination of these. 

The present invention provides a vehicle headlight control which is exceptionally 
discriminating in identifying oncoming headlights and leading taillights in a commercially 
viable system which ignores other sources of light including streetlights and reflections of the 
controlled vehicle*s headlights off signs, road markers, and the like. The present invention 
further provides a sensor having the ability to preselect data from the scene forward of the 
vehicle in order to reduce the input data set to optimize subsequent data processing. The 
invention is especially adapted for use with, but not limited to, photoarray imaging sensors, such 
as CMOS and CCD anays. 

These and other objects, advantages, and features of this invention will become apparent 
upon review of the follov^ng specification in conjunction with the drawings. 

BRI EF DESCRIPTION OF THE PR AWTNnSi 

Fig. 1 is a side elevation of a portion of a vehicle embodying the invention; 

Fig. 2 is a partial side elevation view and block diagram of a vehicle headlight dimming 
control system according to the invention; 

Fig. 3 is a block diagram of the control system in Fig. 2; 

Fig. 4 is a layout of a light-sensing array useful with the invention; 

Fig. 5 is a block diagram of an imaging sensor; 
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Fig. 6 is an alternative embodiment of an imaging sensor; 
Figs. 7a-7d are a flowchart of a control program; 

Figs. 8a-8c are spectral charts illustrating spectra regions useful with the invention; 
Fig. 9 is the same view as Fig. 3 of another alternative embodiment; 
5 Fig. 1 0 is the same view as Fig. 2 of an alternative mounting arrangement; 

Figs. 1 la-1 Ic are views forward of a vehicle illustrating different forms of spatial 
filtering; and 

Figs 12a and 12b are illustrations of use of the invention to detect particular atmospheric 
conditions. 

10 DESCRIPTION OF THE PREF ERRED EMBQDIMHNT 

Referring now specifically to the drawings and the illustrative embodiments depicted 
therein, a vehicle 10 includes a vehicle headlight dimming control 12 made up of an imaging 
sensor module 14 which senses light from a scene forward of vehicle 10, an imaging control 
circuit 13 which receives data from sensor 14, and a vehicle lighting control logic module 16 

15 which exchanges data with control circuit 13 and controls headlamps 18 for the purpose of 

modifying the headlight beam (Figs. 1 and 2). Such control may be a binary control of the aim 
of the beam, such as by switching between lamps or lamp filaments, or may be a continuous 
variation of the aim of a single lamp more or less forward of the vehicle. The control may also 
control the intensity or pattern of the beam. Additionally, the lights of a vehicle equipped with 

20 daytime running lights may be switched between a daytime running light condition and a low- 
beam condition. Vehicle headlight dimming control 12 can perform a wide range of additional 
control operations on the vehicle, including turning the headlights ON and OFF, modifying the 
light intensity of the instrument panel, and providing an input to an electro-optic mirror system. 
Vehicle lighting control logic module 16 receives an input 20 from imaging control 

25 circuit 13. In particular embodiments, such as ones which adjust the state of the headlights 

between continuously variable states, module 16 may supply data to imaging control circuit 13, 
such as the speed of the vehicle, which may be combined with the data sensed by imaging sensor 
14 in establishing the state of headlights 1 8. In the illustrated embodiment, imaging sensor 
module 14 may be fixedly mounted in a housing 28 by a bracket 34 mounted to, or near, the 

30 vehicle's windshield 32. Bracket 34 also moimts an interior rearview minor 30. This is a 

preferred mounting for imaging sensor module 14 because the location within the interior of the 
vehicle substantially eliminates environmental dirt and moisture from fouling the light sensor 
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module. Additionally, the position behind windshield 32, which typically is kept relatively clear 
through the use of washers and wipers and the like, ensures a relatively clear view of the scene 
forward of vehicle 10. Alternatively, imaging sensor module 14 may be mounted within a 
housing 29 of interior rearview mirror 30 facing forward v^th respect to vehicle 10 (Fig. 10). In 
5 such embodiment, control circuit 1 3 may be combined v^th the circuit which controls the partial 
reflectance level of mirror 30 if mirror 30 is an electro-optic mirror such as an electrochromic 
miiTor. Other mounting techniques for sensor module 14 will be apparent to the skilled artisan. 

Imaging sensor module 14 includes an optical device 36, such as a lens, an array 38 of 
photon-accumulating light sensors, and a spectral separation device for separating light from the 
10 scene forward of vehicle 10 into a plurality of spectral bands, such as a filter array 40 disposed 
between optical device 36 and light-sensing array 38. Light-sensing array 38 is described in 
detail in co-pending application Serial No. 08/023,918 filed February 26, 1993, by Kenneth 
Schofield and Mark Larson for an AUTOMATIC REARVIEW MIRROR SYSTEM USING A 
PHOTOSENSOR ARRAY, the disclosure of which is hereby incorporated herein by reference. 
1 5 Light-sensing array 36 includes a plurality of photosensor elements 42 arranged in a matrix of 
columns and rows (Fig. 4). In the illustrated embodiment, an array of 512 rows and 512 
columns of light-sensing pixels, each made up of a photosensor element 42 is utilized. 
However, a greater or lesser number of photosensor elements may be utilized and may be 
arranged in matrix that is laid out in other than columns and rows. Each photosensor element 42 
20 is connected to a common word-line 44. To access the photosensor array, a vertical shift register 
46 generates word-line signals to each word-line 44 to enable each row of photosensor elements 
42. Each column of photosensor elements is also connected to a bit-line 48 which is connected 
to an amplifier 50. As each word-line 44 is accessed, a horizontal shift register 52 uses a line 54 
to output the bit-line signals on consecutive bit lines 48 to an output line 56. In this manner, 
25 each photosensor element 42 may be individually accessed by appropriate manipulation of shift 
registers 46 and 52. Output 56 is supplied to a digital signal processor 13 which is supplied on 
an output 62 as input to control circuit 13 (Figs. 3-5). 

Digital signal processor 13 includes an analog-to-digital converter 58 which receives the 
output 56 of array 36 and converts the analog pixel values to digital values. A digital output 68 
30 of A/D converter 58 is supplied to a taillight detection circuit 76, a headlight detection circuit 78, 
and to ambient sense logic circuit 84. A detection control circuit 72 supplies control and timing 
signals on a line 74 which is supplied to array 38, AJD converter 58 taillight detection circuit 76, 
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headlight detection circuit 78, and ambient sense logic 84. Such signals coordinate the activities 
of these modules and provide any data, from look-up tables provided in control circuit 72, 
needed by each circuit to perfomi its function. For example, control circuit 72 may provide 
intensity threshold levels to taillight detection circuit 76 and headlight detection circuit 78, 
5 Taillight detection circuit 76 detects a red light source having an intensity above a 

particular threshold as follows. For each pixel that is "red," a comparison is made with adjacent 
"green" pixels and "blue" pixels. If the intensity of a red pixel is more than a particular number 
of times the intensity of the adjacent green pixel and adjacent blue pixel, then it is determined 
that the light source is red. If the intensity of the "red" light source is greater than a particular 

10 threshold, an indication is provided at 80. 

Headlight detection circuit 78 detects a vAiiX^ light source having an intensity above a 
particular threshold as follows. A white light is a combination of red, green, and blue 
components. If adjacent "red," "green," and "blue" pixels all exceed a particular threshold, a 
ratio comparison is made of the pixels. If the ratio of the intensity of the adjacent "red," 

1 5 "green," and "blue pixels is within a particular range, such as 20 percent by way of example, 
then a white light source is detected. 

Vehicle headlight dimming control 12 additionally includes an ambient light-sensing 
circuit 84 which receives an input from digital output signal 68. Ambient detection circuit 84 
samples a subset of photosensor elements and detects light levels sensed by the subset over a 

20 long period of time in order to produce significant time filtration. Preferably, the photosensor 
elenxents in the sensed subset include sensors that detect portions of the forward-looking scene 
that are just above the earth's horizon which is more indicative of the ambient light condition. 
Ambient detection circuit 84 produces an indication 88 of ambient light levels which is supplied 
as an input to a lighting control module 90. A high ambient light level may be used by a module 

25 90 to inhibit headlight actuation or to switch headlights 1 8 to a daytime running light mode. 
Ambient detection circuit 84 can, optionally, perform other functions, such as switching the 
daytime ruruiing lights of the vehicle between daytime and nighttime modes, controlling the 
intensity of the vehicle's instrument panel and providing an input to an electro-optic rearview 
mirror system. 

30 Indications 80 and 82 from the light detection units and indication 88 from ambient 

detection circuit 84 are supplied to a lighting control circuit 90 which produces a first indication 
92 that headlights 1 8 are to be switched on, or switched from a daytime ruiming condition to a 
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night mode, and a high-beam enable indication 94 that the headlights may be switched to a high- 
beam state. Vehicle lighting control logic module 16 responds to indications 92 and 94 by 
switching headlights 1 8 to an appropriate mode. An output 96 from module 16 may be provided 
to supply lighting control circuit 90 with information with inspect to vehicle telemetry, steering, 
speed, and any other parameter that may be incorporated into the algorithm to determine the 
state of the headlights of the vehicle. Digital signal processor 13 may be implemented using 
discrete digital circuit modules or with a suitably programmed micro-processor with input and 
output buffers. 

In one embodiment, an imaging sensor module 14a includes a single photosensor array 
38a, one spectral filter array 40a, and one optical device 36a (Fig. 5). In this illustrated 
embodiment, spectral filter array 40a includes alternating spectrum filter elements for exposing 
adjacent pixels to different regions of the electromagnetic spectrum in the red band or green 
band or blue band. This may be accomplished by arranging such filter elements in stripes or by 
alternating filter spectral regions in a manner known in the art. Digital signal processor 13a 
captures a frame of data by enabling photosensor array 38a for a particular exposure period 
during which each photosensor element 42 accumulates photons. In order to detect oncoming 
headlights, digital signal processor 13a enables photosensor array 38a for a first exposure period. 
In order to detect leading taillights, digital signal processor 13a enables photosensor array 38a 
for a second exposure period. Because oncoming headlights have an intensity level that is 
substantially greater than that of leading taillights, the exposure period of the frame in which 
leading taillights is detected is at least approximately ten times the length of the exposure period 
during which oncoming headlights are detected. Most preferably, the exposure period for 
detecting leading taillights is approximately 40 times the exposure period for detecting 
oncoming headlights. In the illustrated embodiment, an exposure period of 0.004 seconds is 
utilized for detecting taiUamps and 0.0001 seconds for detecting oncoming headlamps. The 
exposure period is the time during which each photosensor element 42 integrates photons before 
being read and reset by digital signal processor 13a. Establishing a different exposure period for 
detecting headlights verses taillights facilitates the use of existing and anticipated sensor 
technology by accommodating the dynamic range of such sensor technology. Exposure may 
also be adapUvely established on a priority basis. In one such embodiment, exposure is set to a 
shorter headlight setting. If headlights are detected, the headlights 18 of vehicle 10 are dimmed 
and the exposure period is kept short. If no headlights are detected, the next frame is set to a 
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longer exposure period. This has the advantage of shorter system cycle time as well as a 
reduction in sensitivity to sensor saturation and blooming. In another such embodiment, the 
exposure period is initially set to a long period. If an oncoming headlight is tentatively detected, 
the exposure period could then be switched to a short period to confirm the observation. 
5 Vehicle headlight dimming control 12 carries out a control routine 1 00 (Figs. 7a-7d). At 

the beginning of each pass through the routine, which occurs for every frame captured by the 
imaging sensor, a frame is grabbed at 102 and all of the pixels in the frame are processed as 
follows. Counters used for detecting white headlight sources and red taillight sources are zeroed 
at 104. It is then determined at 106 whether the previously processed frame was for detecting 

10 headlights or taillights. This is determined by looking at a variable "process.tails" which will be 
set to "yes" if the previous frame was processed to detect headlights and will be set to "no" if the 
previous fi^e was processed to detect taillights. If it is determined at 1 06 that the variable 
"process.tails" is set to "yes," the control proceeds to 108 in order to process the next fi^e to 
detect taillights. If it is determined at 106 that the variable process.tails is set to "no," then 

1 5 control passes to 1 09 in order to process the next frame as a headlight detecting frame. 

The taillight detecting frame process begins at 108 by setting the exposure period for the 
imaging sensor module to grab the next frame according to a headlamp exposure level. In the 
illustrated embodiment, the exposure period for detecting headlights is set at 0.0001 seconds. 
Processing of the taillight frame proceeds at 1 10 by examining, for each "red" pixel, whether the 

20 intensity of light sensed by that pixel is greater than a threshold and whether the intensity of 

light sensed by that pixel is greater than a selected number of multiples of the intensity of light 
sensed by an adjacent "blue" pixel and a selected number of multiples of the intensity of light 
sensed by an adjacent "green" pixel. If so, then a "red" counter is incremented at 1 14. 
Preferably, the ratio of red pixel intensity to green or blue pixel intensity is selected as a power 

25 of 2 (2, 4, 8, 16 . . .) in order to ease digital processing. However, other ratios may be used and 
different ratios can be used between red/green and red/blue pixels. In the illustrated 
embodiment, a ratio of 4 is selected based upon ratios established from CIE illuminant charts 
known to skilled artisans. Based upon these charts, a ratio greater than 4 would provide greater 
discrimination. Such greater discrimination may not be desirable because it could result in 

30 failure to identify a leading taillight and, thereby, a failure to dim the headlights of the controlled 
vehicle. After all pixels have been processed, the parameter "process.tails" is set to "no" at 116 
and control proceeds to 1 18 (Fig, 7c). 
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In a similar fashion, processing of a headlight frame begins at 110 by setting the 
exposure period for the imaging sensor module to grab the next frame as a red taillight detecting 
frame. This is accomplished by setting the exposure period of the imaging sensor module to 
0.004 seconds. It is then determined at 120 for each pixel whether an adjacent set of "red," 
5 "green," and "blue" pixels each exceeds a particular threshold and whether the pixel intensity 
levels all fall within a particular range, such as within 20 percent of each other. If all of the red, 
green, and blue pixels exceed a threshold and pass the ratio test, then it is determined that a 
white light source is being sensed and a "white" counter is incremented at 122. After all of the 
pixels in the frame have been processed, the process.tails flag is set to a "yes" state at 124. 

1 0 Control then passes to 118. 

It is determined at 1 18 whether both the "white" and the "red" counters are below 
respective high-beam thresholds. If so, a high-beam frame counter is mcremented and a low- 
beam fi^e counter is set to zero at 1 20. If it is determined at 11 8 that both the "white" and the 
"red" counters are not less than a threshold, it is then determined at 126 whether either the "red" 

1 5 counter or the "white" counter is greater than a respective low-beam threshold. If so, the high- 
beam frame counter is set to zero and the low-beam frame counter is incremented at 128. If it is 
determined at 126 that neither the "red" counter or the "white" counter is greater than the 
respective low-beam threshold, then both the high-beam frame counters and the low-beam frame 
counters are set to zero at 130. 

20 Control then passes to 132 where it is determined if the low-beam frame counter is 

greater than a particular threshold. If so, high-beam enable signal 94 is set to a "low-beam" state 
at 134. Additionally, the low-beam frame counter is set to the threshold level. If it is 
determined at 132 that the low-beam frame counter is not greater than its threshold, it is 
determined at 136 whether the high-beam frame counter is greater than its threshold. If so, high- 

>5 beam enable signal 94 is set to "high-beam" state at 1 38 and the high-beam frame counter is 
reset to its threshold level. 

Control routine 100 provides hysteresis by requiring that a headlight spectral signature or 
a taillight spectral signature be detected for a number of frames prior to switching the headlights 
to a low-beam state. Likewise, the absence of a detection of an oncoming headlight or a leading 

\0 taillight must be made for multiple frames in order to switch from a low-beam to a high-beam 
state. This hysteresis guards against erroneous detection due to noise in a given frame and 
eliminates headlamp toggling when sources are at the fringe of detection range. In the illustrated 
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embodiment, it is expected that a vehicle headlight control system 12 will respond to a change ii 
the state of light sources in the forward field of view of the vehicle in less than 0.5 seconds. An 
additional level of hysteresis may be provided by forcing the headlamps to stay in a low-beam 
state for a given number of seconds after a transition from high beams to low beams. The 
5 reverse would not occur; namely, holding a high-beam state for a particular period to avoid 
annoyance to drivers of oncoming or leading vehicles. 

In the illustrated embodiment, red light sources, which have the spectral signature and 
intensity of taillights, are detected by determining that a "red" pixel, namely a pixel which is 
exposed to light in the visible red band, is both greater than a given multiple of the "green'' and 

10 "blue" adjacent pixels, as well as being greater than a threshold and that white light sources, 

which are the spectral signatures of headlights, are detected by determining that "red," "green," 
and "blue" pixels are both within a particular intensity range of each other as well as being 
greater than a threshold. This double-testing helps to reduce false detection of light sources. 
However, it would be possible to detect red light sources only by looking at the intensity of 

1 5 "red" pixels and to detect white light sources by determining that an adjacent set of "red," 
"blue," and "green"pixels are all above a particular threshold. 

In the illustrated embodiment, spectral filtering is carried out in a manner which exposes 
each photosensing element in the photosensor array to a band of light falling within one of the 
primary ranges of the visible spectrum, namely red, green, or blue as illustrated in Fig. 8a. 

20 However, different bands in the frequency spectrum may be utilized including not only visible 
spectrum bands but invisible spectrum bands including infrared and ultraviolet bands as 
illustrated in Fig. 8b. The band selection could also be chosen from visible spectral regions that 
do not correspond with the primary spectrums. For example, the spectral filter may be selected 
in order to detect at the pixel level red light sources and the compliment of red light sources as 

25 illustrated in Fig. 8c. These binary indications could be utilized to detect red taillights by 
determining that the "red" pixel is greater than a threshold and greater than a number of 
multiples of the intensity sensed by the "red compliment" pixel adjacent thereto. Likewise, a 
white light source indicative of oncoming headlights could be detected by determining that both 
the "red" pixel and the "red compliment" pixel adjacent thereto are both above a particular 

30 threshold and within a particular intensity range of each other. It may also be desirable to select 
bands that fall between primary spectrum regions or any other bands that may be desirable for a 
particular application. 
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Photosensing array 38 may be a charge couple device (CCD) array of the type commonly 
utilized in video camcorders and the like. Alternatively, photosensing array 38 could be a 
CMOS array of the type manufactured by VLSI Vision Ltd. (VVL) in Edinburgh, Scotland. 
Additionally, a hybrid of the CCD and CMOS technology may be employed. Other potentially 
useful photosensing technologies include CID, MOS, photo diodes, and the like. 

In an alternative embodiment, an imaging sensor module 14b includes two or more pairs 
of photosensor arrays 38b (Fig. 6). Each photosensor array 38b has an associated spectral filter 
array 40b and optical device 36b. In this embodiment, each array 38b is operated by digital 
signal processor 58b to have an exposure period that is set for detecting either oncoming 
headlights or leading taillights. In this manner, each frame of the scene captured by each array is 
utilized to detect a particular light source. This is in contrast to light-sensing module 14a in Fig. 
5 in which each light source is detected in alternating frames. Each spectral filter 40b is 
identical, whereby each array 38b is capable of detecting light sources having spectrum 
composition including red, green, and blue regions of the spectrum. However, the spectral filters 
may be custom configured to the particular application. This may result in a homogeneous 
composition or a more complex mosaic, especially where light sources are examined in three or 
more spectral regions. 

In yet an additional single lens system embodiment, an imaging sensor module 14c 
includes three light-sensing arrays (not shown) and a spectral separation device overiying the 
light-sensing arrays which directs spectral bands to different arrays (Fig. 9). An example of such 
spectral separation device is a refracting optical splitter, such as dichroic miirors or prisms. In 
this manner, each light-sensing array detects light in either the red or green or blue region of the 
spectrum. As such, imaging sensor module 14c produces three output signals on a line 64, each 
representing detected light in one of the red or green or blue spectral regions. The output signals 
on line 64 include frame-timing signals which are decoded by digital acquisition circuits 66 
which produces a digital output signal 68' indicative of intensity levels of adjacent red, green, 
and blue pixels. Digital acquisition circuit 66 additionally produces a timing signal output 70 
which is utilized by a detection control circuit 72 in order to supply synchronizing signals, at 74, 
to imaging sensor module 14c and digital acquisition circuit 66. A control and timing signal 86 
is produced by digital acquisition circuit 66 and supplied to detection circuits 76 and 78 and 
ambient detection circuit 84 in order to enable the circuits to distinguish between subsequent 
frames captured by the light-sensing modules. As with previously described embodiments, 
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digital output signal 68' is supplied to taillight detection circuit 76, headlight detection circuit 78, 
and ambient sense logic circuit 84. 

The present invention is capable of identifying point sources of light in any particular 
location within the scene viewed forward of the vehicle. Additional discrimination between 
5 oncoming headlights and leading taillights may be accomplished by taking into account the 

relative location of the source of light within the scene. For example, as best seen by reference 
to Fig. 1 la, particular relationships have been discovered to exist between light sources of 
interest and their spatial location forward of the vehicle. Oncoming headlights and leading 
taillights of interest can be characterized, at least in part, based upon their displacement from the 

10 central axis of the vehicle. On-axis light sources of interest can be at both close and far away 
separation distances. However, off-axis light sources may only be of interest if at a close 
separation distance from the vehicle. Assuming for illustration purposes that headlights and 
taillights are of the same size, headlights and taillights of interest occupy an increasing spatial 
area as they move off axis. Therefore, the resolution required to detect lights of interest may 

15 decrease off axis. Additionally, the fact that close-up off-axis light sources have significant 

spatial area would allow image-processing techniques to be employed to discriminate between 
close-up off-axis light sources of interest and distant off-axis light sources, which are not of 
interest. This may be accomplished through customized optics or other known variations in 
pixel resolution. Furthermore, headlights and taillights of interest are of greater intensity, 

20 because of their closeness, off axis. This allows an increase in intensity detection thresholds off 
axis without missing detection of such light sources. This increase in detection threshold and 
reduction in resolution off axis assists in avoiding false detection of light sources not of interest, 
such as a streetlights, building lights, and the like. 

In order to take into account this spatial differentiation, the present invention 

25 comprehends detecting light sources at a lower threshold centrally of the scene and at a higher 
threshold at the periphery of the scene. This may be accomplished either optically, or 
electronically, or both. Optically, this may be accomplished by providing a non-uniform 
magnification to optical device 36. For example, an optical device may have optical 
magnification at a central portion thereof and an optical attenuation at a peripheral region 

30 thereof. Additionally, optical device 36 may have a relatively wide horizontal field of view and 
a relatively narrow vertical field of view. The narrow vertical field of view would tend to reduce 
the detection of street lights and other overhead light sources. In a preferred embodiment. 
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optical device 36 is a lens that is made from injection-molded plastic. Electronically, such 
spatial differentiation may be accomplished by establishing a higher threshold level for pixel 
intensity detection for pixels located at the periphery of the scene than for pixels located 
centrally of the scene. This would cause centrally positioned light sources to be detected at a 
lower intensity level than sources detected at the periphery of the scene. Such spatial 
differentiation could also be accomplished by a non-symmetrical mapping of light to the sensor 
array, as illustrated in Fig. lib, or by masking portions 98a, 98b, and 98c, at the periphery of the 
scene, as illustrated in Fig. 1 Ic, so that these portions are not sensed at all. Spatial 
differentiation could also be accomplished by providing non-uniform pixel size. 

The present invention is exceptionally sensitive to sources of light having spectral 
signatures of oncoming headlights and leading taillights. By recognizing the spectral signature 
of the light sources, many non-relevant light sources may be ignored. By examining light 
sources pixel-by-pixel, relatively small light sources may be detected at great distances in order 
to dim the headlights well before they become a nuisance to the driver of the vehicle ahead of 
the control vehicle. This is accomplished, according to a preferred embodiment, by utilizing an 
imaging sensor made up of an array of photosensing elements in a compact design which 
responds to light sources in a scene forward of the vehicle. Furthennore, such sensor preferably 
utilizes digital processing techniques which are well adapted for use with custom digital 
electronic circuity, avoiding the expense and speed constraints of general purpose programmable 
microprocessors. 

The present invention takes advantage of the spectral signatures both of light sources 
which must be detected in a headlight dimming control as well as the spectral signatures of light 
sources which must be rejected in a headlight dimming control. For example, federal 
regulations establish specific spectral bands that must be utilized in vehicle taillights; namely 
red. Furthermore, federal legislation prohibits the use of red light sources in the vicinity of a 
highway. Lane markers, signs, and other sources of reflected light are all specified in a manner 
which may be readily identified by spectral signature. Oncoming headlights, according to 
known technology, have a visible spectral signature which is predominantly white light. As 
light source technology evolves, the present invention facilitates detection of other spectral 
signatures of light sources in the future. 

The present invention is capable of utilizing spatial filtering to even further enhance the 
ability to identify light sources. By spatial filtering is meant consideration of not only whether a 
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particular pixel, or pixel group, is detecting a light source having a particular spectral signature, 
but also what adjacent, or closely related, pixels or pixel groups, are detecting. For example, it 
can be concluded that very closely adjacent red and white light sources are not of interest as 
oncoming headlights or taillights. An example where such pattern could be observed is a 
5 streetlight observed with a system having imperfect color correction, which can produce a white 
light surrounded by a red halo. By evaluation of adjacent pixel groups, a closely proximate red 
light source and white light source can be identified as a streetlight and not either a headlight or 
a taillight. 

Pattern recognition may be used to further assist in the detection of headlights, taillights, 

10 and other objects of interest. Pattern recognition identifies objects of interest based upon their 
shape, reflectivity, luminance, and spectral characteristics. For example, the fact that headlights 
and taillights usually occur in pairs could be used to assist in qualifying or disqualifying objects 
as headlights and taillights. By looking for a triad pattern, including the center high-mounted 
stoplight required on the rear of vehicles, stoplight recognition can be enhanced. Furthermore, 

15 object recognition can be enhanced by comparing identified objects over successive frames. 
This temporal processing can yield information on object motion and can be used to assist in 
qualifying or disqualifying objects of interest. 

Spatial filtering can also be useful in identifying atmospheric conditions by detecting 
effects on light sources caused by particular types of atmospheric conditions. One such 

20 atmospheric condition is fog. A bright light source 102 is surrounded by a transition region 104 
between the intensity of the light source and the black background (Fig. 12a). Fog, or fine rain, 
tends to produce a dispersion effect around light sources which causes a series of transition 
regions 104a, 104b . . . 104n which extend further from the light source (Fig. 12b). By placing 
appropriate limits on the size of the transition region, fog or light rain, or a mixture of both, or 

25 other related atmospheric conditions, can be detected. In response to such atmospheric 

conditions, vehicle headlight dimming control 12 may activate fog lights, inhibit switching to 
high beams, or perform other conU-ol functions. Furthermore, fog, or fine rain, can be detected, 
or confirmed, by analyzing the effects of headlights 1 8 in the forward scene as reflected off of 
moisture particles. 

30 Spatial filtering can also be used to detect rain on windshield 32. This may be 

accomplished by performing statistical analyses between a pixel, or pixel group, and adjacent 
pixels or pixel groups. A view forward of a vehicle through a dry windshield would be sensed 
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by an imaging sensor module as continuously varying differences between adjacent pixels, or 
pixel groups, assumed to be under constant illumination from light sources. When, however, a 
droplet of water or a snowflake is on windshield 32, an effect is created which causes a lack of 
continuous variation of differences between adjacent pixels, or pixel groups. This has the 
tendency to reduce the first derivative of the pixel, a condition which can be determined by 
processing. 

Processing can be used to determine the first derivative of an image captured by image- 
sensing module 14 by detemiining a measure of the entropy, or disarray, of a pixel, or pixel 
group, with respect to its neighbors. For example, an approximation of the first derivative for a 
pixel is: 



LiPi-Pjf 



dxy N 
where N=8 and 

where Pi is a given pixel and Pj is one of 8 neighboring pixels. 

It should be apparent to those skilled in the art that the invention is capable of performing 
control functions other than controlling the dimming of the vehicle's headlights. For example, 
spectral signature identifications may be utilized to detect the state of a traffic light to either 
warn the driver that a light has changed from green to yellow to red or to automatically 
decelerate and stop the vehicle. Also, by sensing diat the intensity of a leading taillight has 
abruptly increased, a condition where the leading vehicle is braking may be identified and 
suitable action taken. 

The invention may be utilized to identily particular traffic signs by their spectral 
signature as well as their geometric organization. For example, red octagons may be identified 
as stop signs, yellow triangles as caution signs, and the like. These capabilities are a result of the 
present invention providing a significant reduction in the amount of data to be processed because 
the image forward of the vehicle is captured in a manner which preselects data. Preselection of 
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data is accomplished by configuring the sensor array, including the optics thereof, to consider 
the spatial, as well as the spectral, characteristics of light sources. 

The present invention may be used to determine the environment in which the vehicle is 
operated. For example, a high level of "non-qualified" light sources; namely, light sources that 
are not headlights or taillights, as well as "qualified" light sources can be used to determine a 
measurement of the activity level around the vehicle; namely, that the vehicle is in an urtjan 
environment which may be a useful input for particular control algorithms. This may be 
accomplished as follows. An activity counter is established which represents a total number of 
pixels, or pixel groups, whose red, green, or blue comp)onents exceed a threshold. The threshold 
is set to a relatively low value, namely just above the noise floor. This counter, which registers 
any real detected source, is reset and retabulated every frame, preferably during the exposure 
period for detecting taillights. If the activity counter exceeds a particular value, then a high 
activity environment is detected. One use of this information would be to inhibit the control 
from switching the vehicle's headlights from a low-beam stale to a high-beam state unless a low 
activity condition exists for awhile. The activity counter may be used by the control in 
combination with a low-beam duration counter which records the number of fi-ames that the 
system has been in a low-beam state. It is reset upon system power-up and at every transition 
from the high-to-low beam states. The control may be inhibited fi-om switching the vehicle's 
headlights to the high-beam state unless either the low-beam duration counter exceeds a value or 
the activity coimter indicates a sustained low activity condition. 

The present invention can be used to detect lane markers in order to either assist in 
steering the vehicle or provide a warning to the driver that a lane change is occurring. The 
capability of the invention to detect rain on the vehicle's windshield could be used to control the 
vehicle's wipers both between OFF and ON conditions and to establish a frequency of 
intermittent operation. 

Changes and modifications in the specifically described embodiments can be carried out 
without departing from the principles of the invention which is intended to be limited only by 
the scope of the appended claims, as interpreted according to the principles of patent law 
including the doctrine of equivalents. 
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The embodiments of the invention in which an exclusive property or privilege is 
claimed are defined as follows: 



1 . A vehicle headlamp control, comprising: 

an imaging sensor that senses light in spatially separated regions of a field of view 
forward of a vehicle; and 

a control circuit that is responsive to said imaging sensor in order to determine if 
individual regions of the field of view include light levels having a particular intensity level in 
order to identify light sources of interest and provide a control output to said vehicle that is a 
function of the identification of light sources of interest. 

2. The vehicle headlamp control in claim 1 including an exposure control which determines 
an accumulation period of time said imaging sensor senses light. 

3. The vehicle headlamp control in claim 2 wherein said exposure control defines at least 
two accumulation periods, one of said accumulation periods for detecting oncoming vehicle 
headlights and another of said accumulation periods for detecting loading vehicle taillights. 

4. The vehicle headlamp control in claim 1 wherein said control circuit includes a digital 
processor which provides a digital value of the sensed light level of each region. 

5. The vehicle headlamp control in claim 1 wherein said imaging sensor includes a 
photosensor array. 

6. The vehicle headlamp control in claim 5 wherein said imaging sensor includes a spectral 
separation device for dividing said photosensors into sensors which are each responsive to a 
particular spectral region. 

7. The vehicle headlamp control in claim 6 wherein said spectral separation device divides 
said photosensors into red, green, and blue sensors which sense light in either a red, a green, or a 
blue portion of the spectrum. 
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1 8. The vehicle headlamp control in claim 6 wherein said spectral separation device divides 
said photosensors into red and red compliment sensors which sense light in either a red portion 
of the spectrum or a portion of the spectrum that is outside said red portion of the spectrum. 

1 9. The vehicle headlamp control in claim 1 wherein said imaging sensor includes an optical 
device. 

^ 1 0. The vehicle headlight control in claim 9 wherein said optical device produces a higher 
resolution along an axis of vehicle travel than a resolution off of said axis. 

1 11. The vehicle headlamp control in claim 1 wherein said control circuit switches the vehicle 
headlights between high-beam and low-beam states. 

^ 12. The vehicle headlamp control in claim 1 wherein said control circuit controls the vehicle 
headlights along a continuously variable value. 

^ 13. A vehicle headlight control, comprising: 

a solid-state light sensor array made up of a plurality of sensors arranged in a matrix on 
at least one semiconductor substrate and a spectral separation device to apply particular spectral 
regions to particular ones of said sensors, wherein each of said sensors responds to light in a 
5 particular spectral region; 

an optical device for directing light onto said sensor array in a manner which preserves 
spatial separation of light sources forward of the vehicle; and 

a control circuit that is responsive to said plurality of sensors in order to determine if 
spatially adjacent one or ones of said sensors are sensing light of a particular spectral signature 
10 above a particular intensity level in order to identify light sources that are either oncoming 
headlights or leading taillights to thereby control the vehicle headlights. 

1 14. The vehicle headlamp control in claim 13 wherein said control circuit selects a first light 
exposure period for at least a portion of said array in order to detect oncoming headlights and a 
second light exposure period for at least a portion of said array in order to detect leading 
taillights. 
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15. The vehicle headlamp control in claim 14 wherein the same portion of said array senses 
oncoming headlights and leading taillights and said first and second light exposure periods are 
sequentially varied. 

1 6. The vehicle headlamp control in claim 1 4 wherein different portions of said array sense 
oncoming headlights and leading taillights and each of said portions has one of said light 
exposure periods. 

17. The vehicle headlamp control in claim 14 wherein the length of said second light 
exposure is at least ten times the length of said first light exposure. 

18. The vehicle headlamp control in claim 17 wherein the length of said second light 
exposure is at least approximately 40 times the length of said first light exposure. 

19. The vehicle headlamp control in claim 13 wherein said optical device produces a higher 
resolution along an axis of vehicle travel than a resolution off of said axis. 

20. The vehicle headlamp control in claim 19 wherein said optical device has a 
magnification that is greater centrally of its field of view than at the periphery of its field of 
view. 

21. The vehicle headlamp control in claim 1 3 including an ambient light detector which 
enables said control circuit during low ambient light conditions. 

22. The vehicle headlamp control in claim 21 wherein said ambient hght detector includes a 
portion of said sensors that are time-filtered in order to sense long duration changes in sensed 
light levels. 

23. The vehicle headlamp control in claim 22 wherein said portion of said sensors are 
positioned on said array to receive light from sources close to the earth's horizon. 
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24. The vehicle headlight control in claim 1 3 wherein particular ones of said sensors respond 
to red light in generally a red spectral region and particular ones of said sensors respond to non- 
red light in a spectral region other than said red spectral region. 

25. The vehicle headlight control in claim 24 wherein said control circuit detects leading 
taillights by determining that at least one of said red light responsive sensors is sensing a light 
level that is greater than a particular multiple of light sensed by at least one adjacent non-red 
light responsive sensor and that light level is greater than a particular threshold. 

26. The vehicle headlight control in claim 24 wherein said control circuit detects oncoming 
headlights by determining that at least one of said red light responsive sensors and at least one 
adjacent non-red light responsive sensors are both sensing light levels within a particular ratio of 
each other and greater than a particular threshold. 

27. The vehicle headlight control in claim 1 3 wherein said control circuit detennines if a 
particular light source has a particular spectral signature by comparing levels of light sensed by 
sensors which respond to light in a particular spectral region with levels of light sensed by 
sensors which respond to light in a different spectral region. 

28. The vehicle headlight control in claim 1 3 wherein said control circuit determines if a 
particular light source has a particular spectral signature by comparing levels of light sensed by 
sensors which respond to light in a particular spectral region with a threshold. 

29. The vehicle headlight control in claim 28 wherein said threshold is a function of the 
spatial location of the particular sensor in the array. 

30. The vehicle headlight control in claim 29 wherein said threshold is higher for sensors 
sensing light off a central forward axis of the vehicle than for sensors sensing light along said 
axis. 

3 1 . The vehicle headlight control in claim 28 wherein said control circuit determines if a 
particular light source has a particular spectral signature by comparing levels of light sensed by 
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sensors which respond to light in a particular spectral region with levels of light sensed by 
sensors which respond to light in a different spectral region. 



32. The vehicle headlight control in claim 1 3 wherein said solid-state light sensor and said 
optical device are positioned in a housing behind the vehicle's windshield. 

33. The vehicle headlight control in claim 32 wherein said housing is behind a portion of the 
vehicle's windshield swept by wipers. 

34. The vehicle headlight control in claim 33 wherein said housing is mounted to a bracket 
supporting a rearview mirror. 

35. The vehicle headlight control in claim 32 wherein said housing is an interior mirror 
housing. 

36. A method of detecting light sources of interest forward of a vehicle in order to control the 
headlights of that vehicle, including: 

providing a solid-state photosensor array made up of a plurality of sensors arranged in a 
matrix and mapping particular spatially arranged portions of a field of view forward of the 
vehicle onto said array in a manner which generally preserves the spatial arrangement of the 
field of view; 

restricting light received by each sensor to a particular spectral region so that each sensor 
responds to light having a particular spectral range; and 

evaluating light levels sensed by each sensor in order to establish a spectral signature of 
light sources in said field of view in order to identify light sources of interest in said field of 
view. 

37. The method in claim 36 wherein said comparing includes exposing at least a portion of 
said sensors for a first exposure period in order to identify spectral signatures of headlights and 
exposing at least a portion of said sensors for a second exposure period in order to identify 
spectra signatures of taillights. 
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38. The method in claim 36 fiirther including reading individual light intensity levels of each 
of said sensors. 



39. The method in claim 38 including digital processing of said individual light intensity 
levels. 

40. The method in claim 39 including digital processing using custom digital circuitry. 

41 . The method in claim 36 wherein said directing light includes a non-linear mapping of the 
field of view forward of the vehicle onto said array. 

42. The method in claim 36 wherein said mapping includes masking portions of the field of 
view forward of the vehicle. 

43. The method in claim 36 wherein said comparing light levels includes comparing light 
levels with thresholds, said thresholds being different for particular ones of said sensors. 

44. The method in claim 43 wherein said thresholds are greater off a central forward axis of 
the vehicle than on said axis. 

45. A method of detecting light sources of interest forward of a vehicle in order to control the 
headlights of that vehicle, including: 

providing a solid-state photosensor array made up of a plurality of sensors arranged in a 
matrix and mapping particular spatially arranged portions of a field of view forward of the 
vehicle onto said array in a manner which generally preserves the spatial arrangement of the 
field of view; 

evaluating light levels detected by each sensor in order to identify light sources of 
interest at least in part as a function of a spatial distribution of each light source in said field of 
view. 

46. The method in claim 45 wherein said evaluating includes establishing a different sensing 
resolution along an axis of vehicle travel than off of said axis. 
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47. The method in claim 46 wherein the sensing resolution along said axis is greater than off 
of said axis. 



48. The method in claim 45 including providing an optic device which provides a non- 
uniform magnification of said field of view. 

49. The method in claim 45 wherein said solid-state photosensor array has sensor sizes 
which are non-uniform in said matrix. 



50. The method in claim 45 wherein said identifying light sources of interest includes 
determining if each sensor is sensing a light level above a threshold level. 

51 . The method in claim 50 wherein said threshold is non-uniform for said sensors. 

52. The method in claim 5 1 wherein said threshold is lower for sensors sensing portions of 
said field of view along an axis of vehicle travel than for sensors sensing a portion of said field 
of view off of said axis. 



53. The method in claim 45 including restricting light received by each sensor to a particular 
spectral region so that each sensor responds to light having a particular spectral range. 
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